Successful implementation of a rater training program for medical students to evaluate simulated pediatric emergencies.

Introduction: Simulation-based training is increasingly used in pediatrics to teach technical skills, teamwork, and team communication, and to improve potential deficiencies in pediatric emergency care. Team performance must be observed, analyzed, and evaluated by trained raters. The structured training of medical students for the assessment of simulated pediatric emergencies has not yet been investigated. Methods: We developed a rater training program for medical students to assess guideline adherence, teamwork, and team communication in simulated pediatric emergencies. Interrater reliability was measured at each training stage using Kendall tau coefficients. Results: In 10 out of 15 pairs of raters interrater reliability was moderate to high (tau>0.4), whereas it was low in the remaining 5 pairs of raters. Discussion: The interrater reliability showed good agreement between medical students and expert raters at the end of the rater training program. Medical students can be successfully involved in the assessment of guideline adherence as well as teamwork and team communication in simulated pediatric emergencies.

Sure, or unsure? Measuring students' confidence and the potential impact on patient safety in multiple-choice questions.

BACKGROUND: Multiple-choice questions (MCQs) provide useful information about correct and incorrect answers, but they do not offer information about students' confidence. METHODS: Ninety and another 81 medical students participated each in a curricular neurology multiple-choice exam and indicated their confidence for every single MCQ. Each MCQ had a defined level of potential clinical impact on patient safety (uncritical, risky, harmful). Our first objective was to detect informed (IF), guessed (GU), misinformed (MI), and uninformed (UI) answers. Further, we evaluated whether there were significant differences for confidence at correct and incorrect answers. Then, we explored if clinical impact had a significant influence on students' confidence. RESULTS: There were 1818 IF, 635 GU, 71 MI, and 176 UI answers in exam I and 1453 IF, 613 GU, 92 MI, and 191 UI answers in exam II. Students' confidence was significantly higher for correct than for incorrect answers at both exams (p < 0.001). For exam I, students' confidence was significantly higher for incorrect harmful than for incorrect risky classified MCQs (p = 0.01). At exam II, students' confidence was significantly higher for incorrect harmful than for incorrect benign (p < 0.01) and significantly higher for correct benign than for correct harmful categorized MCQs (p = 0.01). CONCLUSIONS: We were pleased to see that there were more informed than guessed, more uninformed than misinformed answers and higher students' confidence for correct than for incorrect answers. Our expectation that students state higher confidence in correct and harmful and lower confidence in incorrect and harmful MCQs could not be confirmed.

Does quantity ensure quality? Standardized OSCE-stations for outcome-oriented evaluation of practical skills at different medical faculties.

BACKGROUND: Practical skills are often assessed using Objective Structured Clinical Skill Exams (OSCE). Nevertheless, in Germany, interchange and agreement between different medical faculties or a general agreement on the minimum standard for passing is lacking. METHODS: We developed standardized OSCE-stations for assessing structured clinical examination of knee and shoulder joint with identical checklists and evaluation standards. These were implemented into the OSCE-course at five different medical faculties. Learning objectives for passing the stations were agreed beforehand. At each faculty, one reference examiner scored independently of the local examiner. Outcome of the students at the standardized station was compared between faculties and correlated to their total outcome at the OSCE, to their results at the Part One of the National Medical Licensing Examination as a reference test during medical studies and to their previous amount of lessons in examining joints. RESULTS: Comparing the results of the reference examiner, outcome at the station differed significantly between some of the participating medical faculties. Depending on the faculty, mean total results at the knee-examination-station differed from 64.4% to 77.9% and at the shoulder-examination-station from 62.6% to 79.2%. Differences were seen in knowledge-based items and also in competencies like communication and professional manner. There was a weak correlation between outcome at the joint-examination-OSCE-station and Part One of the National Medical Licensing Examination, and a modest correlation between outcome at the joint-examination-station and total OSCE-result. Correlation to the previous amount of lessons in examining joint was also weak. CONCLUSION: Although addressing approved learning objectives, different outcomes were achieved when testing a clinical skill at different medical faculties with a standardized OSCE-station. Results can be used as a tool for evaluating lessons, training and curricula at the different sites. Nevertheless, this study shows the importance of information exchange and agreement upon certain benchmarks and evaluation standards when assessing practical skills.

Feedback promotes learning success! Which kind of feedback for the faculty is given by an interdisciplinary OSCE with focus on decision-making?

Clinical skills such as history taking, diagnostic reasoning, therapy planning, and giving advice are even more complex than practical skills like lung auscultation and have to be applied in complex clinical situations. We judged this competence in an interdisciplinary formative OSCE conducted with students of Marburg University. Results of 218 students passing 643 OSCE stations composed of 37 different scenarios were analyzed. As a competence based examination that reflects the practical skills gained during clinical training, the here presented analysis serves also as a feedback instrument for clinical teachers, their respective disciplines and the medical faculty as a whole.

Teaching differential diagnosis in primary care using an inverted classroom approach: student satisfaction and gain in skills and knowledge.

BACKGROUND: Differential diagnosis is a crucial skill for primary care physicians. General practice plays an increasing important role in undergraduate medical education. Via general practice, students may be presented with an overview of the whole spectrum of differential diagnosis in regard to common symptoms encountered in primary care. This project evaluated the impact of a blended learning program (using the inverted classroom approach) on student satisfaction and development of skills and knowledge. METHODS: An elective seminar in differential diagnosis in primary care, which utilized an inverted classroom design, was offered to students. Evaluation followed a mixed methods design: participants completed a pre- and post-test, a questionnaire, and a focus group discussion. Interviews were transcribed verbatim and answers were grouped according to different themes. Test results were analysed using the Wilcoxon matched-pairs signed-ranks test. RESULTS: Participants (n = 17) rated the course concept very positively. Especially the inverted classroom approach was appreciated by all students, as it allowed for more time during the seminar to concentrate on interactive and practice based learning. Students (n = 16) showed a post-test significant overall gain in skills and knowledge of 33%. CONCLUSIONS: This study showed a positive effect of the inverted classroom approach on students' satisfaction and skills and knowledge. Further research is necessary in order to explore the potentials of this approach, especially the impact on development of clinical skills.

Differential diagnosis in primary care: conception and implementation of a new elective seminar - An experience report.

Primary care is in a unique position to teach the broad spectrum of differential diagnoses. We developed and piloted a new elective seminar 'Differential Diagnosis in Primary Care'. With the help of simulation patients, training models, interactive small group work, and short lectures we addressed common complaints presented in the daily routine of primary care like vertigo, dyspnoea, chest or abdominal pain. We put a special focus on the diagnostic accuracy of history and physical examination. The final examination was conducted as an objective structured clinical examination.